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ABSTRACT 

Both educational vouchers and class size reduction are high- 
profile proposals for improving education. While the body of research on 
vouchers is small and unconvincing, the research on smaller classes is 
abundant and compelling. Researchers have been able to compare the impact of 
both of these policy alternatives on student performance. Their comparison 
shows that smaller classes far outpace vouchers in raising student test 
scores. A recent study on voucher students in various cities found that while 
African American voucher students outscored those who remained in public 
schools, neither white nor Hispanic voucher students showed any academic 
gains over their public school peers. The researchers concluded that there 
was no overall private school impact on the student test scores of those 
using vouchers. Other studies have also failed to buttress the case being 
made by voucher supporters (e.g., studies of the Cleveland and Milwaukee 
voucher programs) . Though voucher supporters claim that public schools will 
not improve without competition, last year, numerous large, urban districts 
raised achievement scores without the presence of publicly funded vouchers. 
Research indicates that smaller classes are more effective at raising student 
achievement than are voucher programs. (Contains 42 endnotes.) (SM) 
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arlier this year, President Bush signed the reauthorization of the Elementary and 
Secondary Education Act (ESEA) or what he called the No Child Left Behind 
Act. Interestingly, one term appears more than 100 times in ESEA: “scientifically 
based research.”^ The goal of ESEA’s bipartisan supporters, explained recently in 
an Education Week article, is to “base school improvement efforts less on intuition and 
experience and more on research-based evidence.”^ As the reform debate continues, it is 
worth examining two distinct proposals — private-school vouchers and class-size reduction — 
and what the research says about their impact on student achievement. 

The U.S. Supreme Court is soon expected to issue a ruling that decides whether the 
Cleveland voucher program is constitutional. At least as important as whether vouchers are 
constitutionally sound is the question of whether vouchers are educationally sound. What 
does the research say about vouchers? And are there more sensible and less divisive 
alternatives — such as reducing class sizes in public schools — ^that can help low-achieving 
students? 

Both vouchers and class-size reduction are high-profile proposals. But relying on 
“scientifically based research” to assess these two proposals reveals some major distinctions. 
While the body of research on vouchers is both small and unconvincing, the corresponding 
research on smaller classes is both abundant and compelling. Most significantly, researchers 
have been able to compare the impact that both of these policy alternatives have on student 
performance. This comparison shows that smaller classes far outpace vouchers in raising 
student tests scores.^ 

Vouchers: What the Most Recent Research Shows 

This week. Harvard University professor Paul E. Peterson is addressing a Heritage 
Foundation audience to discuss his research on voucher students in various cities, including 
New York City. Peterson’s message is likely to resemble the presentation that he and 
University of Wisconsin-Madison professor William G. Howell gave only a few weeks ago 
at a Brookings Institution symposium, reviewing a Peterson-Howell report on a three-year 
study of voucher students. The three-year study compared a “treatment” group (students who 
used a voucher to switch to a private school) with a “control” group (students who attended 
public schools). 

In its findings, the Peterson research team reported that African-American voucher students 
consistently outscored those who remained in public schools. One newspaper trumpeted the 
Peterson-Howell research in an article headlined: “Scores of blacks rise with vouchers.”"* But 
this upbeat coverage — and what is likely to be a similar spin by pro-voucher groups in the 
months ahead — is unjustified by the data, nor does it answer several questions raised by the 
data. In fact, Peterson and Howell specifically admitted that their three-year findings should 
not be used as an argument in favor of a “large-scale voucher program” serving all children 
in an urban school system.^ 
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While voucher supporters are likely to focus on the three-year data showing gains for 
African-American voucher students, there are nagging concerns and questions about the 
Peterson-Howell data. For example, there were large fluctuations among subgroups and 
across the various years of the study, even in New York City. It is also worth noting that 
neither white nor Hispanic voucher students showed any academic gains over their public 
school peers. Additionally, a review of the New York City data reveals that reading scores 
actually declined for all groups of Hispanic voucher students, 
except for those in 7th grade. Peterson and Howell have 
made considerable effort to explain why black scores 
improved and why white and Hispanic scores did not — but to 
no avail.^ All of these considerations help to explain why 
Peterson and Howell concluded that there is “no overall 
private school impact of switching to a private school on 
student test scores 



»7 



Finally, this isn’t the first time a Peterson-led voucher study 
yielded a host of questions or concerns. The conclusions 
drawn by Peterson from a voucher study released in August 
2000 were seriously challenged. (The 2000 data were part of 
the overall three-year study that Peterson and Howell recently 
released.) Researchers Alex Molnar and Charles Achilles 
raised concerns about the August 2000 data, warning that the 
Peterson team’s use of averaged results “may make the 
achievement impact reported appear more generalized than it 

is.”^ And Mathematica Policy Research, one of the partners in the August 2000 study, was 
so disturbed by the conclusions drawn by the Peterson team that the firm took the 
extraordinary step of issuing a press statement entitled “Voucher Claims of Success Are 
Premature in New York City.” Referring to the August 2000 data, Mathematica cautioned 
policymakers against “setting policy based on the overall modest impacts on test scores.”^ 
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Findings on Cleveland and Milwaukee Voucher Plans 

Over the past few years, other research and analyses of voucher programs have failed to 
buttress the case being made by voucher supporters. Last fall, the U.S. General Accounting 
Office reviewed state evaluations and found little or no difference between the academic 
achievement of voucher students and public school students in Cleveland and Milwaukee — 
the two major urban school systems with publicly funded voucher programs. 

Indiana University researcher Kim Metcalf, who has spent several years studying the 
Cleveland program, released a report last year comparing groups of voucher students and 
public school students from the time they entered first grade through the end of second grade. 
While voucher students had higher total test scores entering first grade, this advantage 
quickly began to erode. Over this two-year period, the report revealed that the public school 
students demonstrated average learning gains that were greater in language, reading and 
math than the voucher students. 
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Voucher supporters have cited isolated data from last year’s Indiana University report, 
claiming that Metcalfs research proves that vouchers boost academic performance. Yet 
Metcalf himself wrote that the analysis of student test results from voucher schools and 
public schools “presented no clear or consistent pattern tha[t] can be attributable to [voucher] 
program participation.” Echoing this view, the Ohio Department of Education summed up 
the study in distinctly lukewarm terms, noting that voucher students “perform at a similar 
academic level as public school students. 

The Milwaukee voucher program has received only one comprehensive state evaluation, 
conducted in 1995 by a University of Wisconsin-Madison team led by professor John 
Witte.^^ Reviewing the voucher program’s first five years, Witte found no appreciable 
academic gains in reading and math from vouchers. He also observed that the attrition rates 
for voucher students were high, especially in the first two years. Using Witte’s data, 
a research team led by Peterson employed different assumptions and statistical techniques, 
claiming that there was a statistically significant gain for voucher students in the third and 
fourth years of the program. But this finding was disputed by many in the research 
community, who argued that by the third year the control and experimental groups were not 
comparable. The annual attrition rate (about 30 percent) — consisting primarily of students 
doing poorly in the voucher program — ensured that those students who remained were an 
academically superior subset, not a random sample.^^ Other aspects of the methodology used 
by the Peterson team to re-analyze the Milwaukee data have been criticized, including the 
Peterson team’s reliance, in some cases, on tiny samples — in one instance, a sample of 26 
students. The Peterson team’s re-analysis was described by Witte as a “confusing, tortured 
effort,” and even the pro- voucher Wall Street Journal wrote that Peterson was “loose with his 
claims.”^^ 

Since the 1995 state evaluation, voucher supporters have shown no enthusiasm for new 
efforts to examine the program’s impact on student achievement. In fact, after the lackluster 
results of this evaluation were released, Wisconsin legislators eliminated provisions calling 
for future academic evaluations of the program.^^ Since then, the Legislature has provided 
only for a single audit by the state’s Legislative Audit Bureau in the year 2000. This audit 
observed: “Some hopes for the program — most notably, that it would increase participating 
students’ academic achievement — cannot be documented, largely because uniform testing is 
not required in participating schools.”^^ 

Some voucher supporters have cited research by Princeton University’s Cecilia Rouse that 
reported math gains for Milwaukee voucher students.^^ Yet, the findings Rouse cited were 
only for the subgroup of students who were in the voucher program over a four-year period. 
As noted earlier, student attrition rates come into play because Witte found that “voucher 
students who lefttht [Milwaukee] program for various reasons had lower test scores than 
those who continued to participate [emphasis in original]. Clearly, a full and accurate 
assessment of voucher schools considers not simply those students who use a voucher and 
remain in the voucher school, but, rather, all students who entered the voucher program. In 
simple terms, students who do well in voucher schools are more likely to stay — those doing 
poorly are more likely to leave or drop out. Additionally, Rouse found that “the [voucher] 
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effects on the reading scores are as often negative as positive and are nearly always 
statistically indistinguishable from zero.”^"^ 

The Voucher ‘Competition’ Myth 

Voucher supporters such as researcher Jay Greene claim that vouchers have a positive impact 
on public school students because the threat they pose leads public schools to improve. In 
a February 2001 report, Greene asserted that the ‘‘Florida A+” voucher program led to public 
school gains. But researchers at Rutgers University and the University of Colorado at 
Boulder identified serious flaws in Greene’s analysis.^^ Stanford University professor Martin 
Camoy found that under the accountability system that Florida created before vouchers 
existed, student improvement was greater than after the so-called ‘voucher threat’ was 
introduced.^^ 

Greene also neglected to consider the significant impact of extra resources, both state and 
local, which were directed towards Florida’s ‘F’-rated public schools. These resources 
enabled the schools to extend the school day, week, and year, as well as strengthen 

professional development for teachers. These elements — 
combined with accountability measures — may well have been the 
real cause of improvements in these Florida public schools.^^ 

While pro-voucher forces claim that public schools won’t improve 
without “competition” from voucher programs, the evidence 
dispels this myth. In fact, public school districts in Los Angeles, 
Baltimore, Dallas, Portland, Minneapolis, San Diego, Birmingham 
and Seattle raised both their reading and math scores last year in 
every grade tested — and each of these urban districts did so 
without the presence of a publicly funded voucher program.^^ 
Indeed, Greene’s own research leads to the conclusion that 
accountability, testing, and increased resources led to public school 
improvement in Texas, a state which has no publicly funded 
voucher program.^^ 

Voucher supporters also cite Harvard University researcher 
Caroline Hoxby’s finding that competition from private schools 
spurs improvements in public schools. But Duke University 
professor Helen Ladd and other analysts have questioned Hoxby’s 
conclusions. In a study published earlier this year, Ladd observed that other researchers 
“have used better data and alternative methods and have found no positive effects on public 
school achievement from competition from private schools. 

Class Size Reduction: What We Know 

In stark contrast to vouchers, the research supporting the benefits of class-size reduction is 
both ample and compelling. Indeed, a considerable body of research demonstrates that 
significantly reducing class sizes in the early elementary grades has a major impact in 
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helping to close the achievement gap between white and minority students. This finding is 
supported by one of the most large-scale, comprehensive studies ever conducted in 
education: the Tennessee project called Student-Teacher Achievement Ratio or STAR. The 
highly respected Harvard statistician Frederick Mosteller has called STAR ‘‘one of the most 
important educational investigations ever carried out.”^^ 

In an evaluation involving more than 1 1,000 students, STAR researchers compared the 
progress of students who were in smaller K-3 classes in 1985-89 (pupil-teacher ratios of 14- 
16 to 1) to students who attended regular-sized classes. Researchers found that smaller-class 
students outperformed their peers in regular-sized classes during those years.^^ More 
significantly, however, the smaller-class students continued to outpace their peers in math, 
reading and science for many years to come — even long after returning to regular-sized 
classes in later years. In fact, the gap in test scores between students in the smaller classes 
and the regular classes increased over time. 

STAR researchers also found that the black-white gap in taking college-preparatory exams 
was cut in half for those minorities who had been in smaller classes.^^ Smaller-class students 
were not only more likely to take college-prep exams, but they also scored higher on these 
exams. Jeremy Finn, a professor at the State University of New York, has observed that the 
STAR research “leaves no doubt that small classes have an advantage over larger classes” in 
raising student achievement.^^ 

The benefits of significant class-size reduction have also been demonstrated in other states. 
Started in 1996, Wisconsin’s Student Achievement Guarantee in Education (SAGE) is 
a statewide class-size reduction program that has enjoyed strong bipartisan support and is 
targeted to low-income students in grades K-3. SAGE provides participating schools with 
$2,000 per student to reduce classes to pupil-teacher ratios of 15-1. The program requires 
participating schools to hold extended hours and provide community services to district 
residents. SAGE guidelines also require the development of rigorous curriculum and staff 
development. In the 2001-02 school year, more than 81,000 students statewide are 
participating in SAGE.^^ 

There is extensive research-based evidence supporting SAGE’s success in helping to 
improve student performance. In an evaluation of SAGE and comparison schools, 29 of the 
top 30 classrooms as measured by student achievement in language arts, reading and math 
were SAGE classrooms. The achievement gap in language arts and math between African- 
American and white first-grade students was reduced in SAGE classrooms while it increased 
in comparison schools. Black second- and third-grade students in SAGE schools scored 
higher on every test than their black peers in the comparison schools.^^ Results from the 
recently released 5th-year evaluation of SAGE reinforce these findings.^* The intensive and 
ongoing evaluations of SAGE by Wisconsin officials stand in stark contrast to the 
Milwaukee voucher program, which was the subject of only one state evaluation — now seven 
years old. 

Moreover, the SAGE findings are consistent with research obtained on the impact of class- 
size reduction in other states. For example, smaller classes were identified in a RAND study 
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as one of the “major contributions” to Texas’ significant achievement gains during the 
1990s.^^ 



Class-Size Reduction Versus Vouchers 



How does class-size reduction compare with school vouchers? It’s a question that we can 
answer with surprising clarity thanks to a growing body of research. 



Princeton University researcher Cecilia Rouse, whose findings have been cited by voucher 
supporters, conducted a study in 1998 comparing Milwaukee’s voucher schools with the 
city’s P-5 schools — public schools with small class sizes and additional targeted funding 
(similar to SAGE). “The results suggest,” Rouse concluded, “that students in P-5 schools 
have math test score gains similar to those in the [voucher] 
schools, and that students in the P-5 schools outperform students 
in the [voucher] schools in reading.” Rouse went on to explain: 

“Given that the pupil-teacher ratios in the P-5 and [voucher] 
schools are significantly smaller than those in the other public 
schools, one potential explanation for these results is that 
students perform well in schools with smaller class sizes 
[emphasis in original]. In other words, improved test scores 
for some voucher students may have been the result of attending 
smaller classes. 
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Princeton University researchers Alan Krueger and Diane 
Whitmore compared the effect of attending a smaller class to the 
effect of receiving a private-school voucher. Despite the serious 
questions raised about the Peterson team’s August 2000 voucher 
study, Krueger and Whitmore used the study’s data on African- 
American voucher students for the sake of comparison. (Keep in 
mind, in the August 2000 study African-Americans were the only subgroup of voucher 
students who showed significant gains.) Even in this context, Krueger and Whitmore found 
that black students who had attended small classes “improved their test performance by 
around 50 percent more than the gain experienced by black students who attended a private 
school as a result of receiving a voucher . . 



Although, in statistical terms, class size doesn’t emerge as a determining factor in the 
African-American gains cited in Peterson-Howell’s three-year evaluation of voucher 
students, it is clear that these voucher students were in smaller schools with smaller class 
sizes, and more after-school and tutorial programs."^^ 

Indeed, this is a powerful irony. The African-American voucher students were learning in 
the very educational climate that many policy analysts have long sought for public schools — 
a climate that is incredibly difficult to create when a state diverts substantial tax dollars to 
vouchers. 
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